Lagrangian Transformation and Interior Ellipsoid Methods in Convex Optimization

The rediscovery of the affine scaling method in the late 80s was one of the turning points which led to a new chapter in Modern Optimization the Interior Point Methods (IPMs). The purpose of this paper is to show the intrinsic connections between Interior and Exterior Point methods (EPMs), which have been developed during the last 30 years. A class Ψ of smooth and strictly concave functions ψ : R → R with specific properties is used to transform terms of the classical Lagrangian associated with constraints. 1 The transformation is scaled by a positive scaling parameter. Lagrangian Transformation (LT) methods alternate minimization of the transformed Lagrangian in primal space with Lagrange multipliers update, while the scaling parameter can be fixed or updated from step to step. Our main focus is the equivalence of the Primal Exterior LT and the Dual Interior Ellipsoid Methods. First, we show that the general LT method is equivalent to an Interior Prox method with Bregman type distance function for the dual problem. The distance function is generated by the dual kernel φ = −ψ∗, where ψ is Fenchel conjugate of ψ ∈ Ψ Then, we prove that the Interior Prox is equivalent to an Interior Quadratic Prox in the from step to step rescaled dual space which, in turn, is equivalent to an Interior Ellipsoid Method for the dual problem. Using the equivalence, we prove convergence of the dual sequence in value for any well defined dual kernel. Using the rescaling technique, we establish the complexity of IEM in terms of the number of steps required for finding an ǫ−approximation to the optimal solution in value for well defined kernels φ ∈ Φ. The LT method with truncated MBF transformation which has been recently considered in [17], leads to the so-called MBF method, which is equivalent to Interior Prox with Bregman distance. The MBF kernel is a self-concordant function on R++, therefore Bregman distance is a self-concordant function on Rq++ and the corresponding Interior Ellipsoids are, in fact, Dikin’s ellipsoids (see [20], [21]). For LP calculations, the LT method with MBF transformation resembles I. Dikin’s affine scaling method for the dual LP.